DOCUMENT RESUME 



ED 466 266 



JC 020 472 



AUTHOR 

TITLE 

INSTITUTION 
PUB DATE 
NOTE 



AVAILABLE FROM 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Piland, William E.; Graham, Martha A.; Hess, Shelly L.; 

Ford, Michael B. 

Southwestern College- -Achieving Institutional Mission: 
Universal Program Review. 

Southwestern Coll., Chula Vista, CA. 

2001-05-00 

19p. ; Paper presented at the Annual Meeting of the Research 
and Planning Group for California Community Colleges (39th, 
Lake Arrowhead, CA, May 2-4, 2001) . 

For full text: http://www.rpgroup.org. 

Reports - Descriptive (141) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Associate Degrees; Basic Skills; Community Colleges; 
Education Work Relationship; ^Educational Assessment; 
Educational Finance; *Educational Objectives; *Institutional 
Mission; Outcomes of Education; Student Needs; *Transfer 
Programs; Two Year Colleges 
♦Southwestern College CA 



ABSTRACT 



This is a report on the universal program review developed 
and administered by Southwestern College (SWC) (California) to help achieve 
its institutional mission and meet accreditation standards. The program 
review was designed by the Achieving Institutional Mission (AIM) committee, 
which consisted of SWC faculty and administrative staff members. The 
committee established five goals for the universal program review: (1) to be 

supported by every segment of the college; (2) to be linked to the college 
budget plans; (3) to establish common and fair criteria for assessing every 
college program; (4) to establish specific criteria to recognize the unique 
and distinct roles of every program; and (5) to establish a review schedule 
that would evaluate every academic program, student service, and 
administrative unit over a five-year period. The review process began in fall 
1999 and completed two cycles. Overall, the review is designed to improve 
college programs and services, to support student learning, to improve 
student goal attainment, and provide evidence of institutional effectiveness 
for accreditation. The program review also acknowledges college 
accomplishments and helps note areas of strength and areas in need of 
improvement. Appendices provide a seven-year program review cycle chart, a 
budget process flowchart, and a list of all SWC academic programs, student 
service departments, and administrative units. (MKF) 
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Southwestern College 

Achieving Institutional Mission: Universal Program Review 

Abstract 



“Southwestern College is committed to meeting the educational goals of its students in an 
environment that promotes intellectual growth and develops human potential.” As college 
employees, we mouth the words of this carefully crafted mission statement, but how do we know 
whether we are achieving our mission? 

Southwestern College had a long history of routinely reviewing its academic programs. 
Student services were also periodically reviewed according to a schedule established by the 
various units. As we at the College took a fresh look at how well we were fulfilling the student- 
centered mission, we realized that several gaps existed in the evaluation process. First, the 
criteria for academic program review had not been examined in five years. Also, no common 
criteria existed to ensure that student services components were consistently reviewed, nor was 
the schedule for student services review firmly established. Finally, neither component of the 
review process was tied in any way to funding or to other College processes. 

Perhaps the greatest gap was the lack of a process to review administrative functions. 
Neither staff sectors (like Human Resources or Fiscal Affairs) nor the administrative functions of 
academic and student services units, such as division offices, had ever participated in review. 
Moreover, there had been no structured attempt to demonstrate how any of the various college 
components support the college mission. 

Over the course of a full academic year, a task force comprised of administrative, 
classified, and faculty representatives worked to design a universal program review process. The 
goal was to design a process that would (1) be supported by every segment of the college; (2) be 
linked to the budget process; (3) establish common criteria for reviewing each of the units fairly 
and consistently; (4) establish specific criteria to recognize the unique roles each unit plays in the 
college; and (5) establish a review schedule that would provide for every academic program, 
student service, and administrative unit to be reviewed over a five-year period. 

This presentation, given at the 2001 Program Review Group Annual Conference, covered 
the scope of this project, from design to implementation, focusing on the role of Institutional 
Research in providing consultation and information both to the process and to the units 
participating in the review. 
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Southwestern College 

Achieving Institutional Mission: Universal Program Review 



Introduction 

For several years, the academic programs/disciplines at Southwestern College have 
undertaken a self-study review to determine their effectiveness in delivering instruction and 
affecting student outcomes. The early days of the review, the mid-1980s, were a time of 
statewide fiscal constraint; the results of program/discipline review were utilized to determine 
which programs/disciplines were discontinued and which were retained. Improvement of 
program/discipline quality was a secondary concern. 

Following that early, somewhat negative experience, a committee of faculty and 
administrators undertook the task of reshaping academic program/discipline review into a 
process that would highlight both the positive aspects and the needs of the academic 
programs/disciplines, with the primary goal being to continuously improve the quality of the 
offerings. The Curriculum and Instruction (C&I) Committee, a shared governance body, created 
Subcommittee C to develop and oversee the review process. In spring 1988, the subcommittee, 
in conjunction with the office of Institutional Research, designed a five-year review process. 

This process would provide for approximately one-fourth of the programs/disciplines being 
reviewed each year for four years; the fifth year would be used to review the process and to 
determine modifications for the next cycle. The academic programs/disciplines have since 
completed two review cycles, with 1999-2000 being the year to review the process. 

A systematic, universal review of all programs and services began in fall 1999. 
Superintendent/President Serafin A. Zasueta, Ph.D., convened an interested group of faculty, 
staff, and administrators to discuss the need for a comprehensive review process. The college 
had received a Title III grant four years prior, and one of its major objectives was the revamping 
of the academic program review process. That requirement, coupled with the fact that the 
academic review process was in the year for reviewing its procedures, made the discussion 
timely. The group quickly reached consensus on a plan that would expand the existing process 
to include all academic, student, and administrative programs and services and also agreed that 
the program review process should be integrated with other campus processes, such as 
accreditation, planning, and budget development. 

Campus Issues 

As can be expected, a comprehensive undertaking to examine and evaluate every 
segment of an institution brought forth many issues expressed by those segments. At 
Southwestern College, there were seven major issues that had to be addressed to ensure success 
in the process. These were: 

• Expansion of the program review process 

• Design of the process by representative groups 

• Relationship of the process to the College mission 

• Program improvement as a result of the process 

• Relationship of the program review process to other College processes 
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• Link between the program review process and the funding process 

• Equitable scheduling of units’ reviews 

The following sections show how the current program review process was designed to 
address each of these campus concerns. 

Expansion of the program review process. For several years, each academic program 
had undergone a self-study “Discipline Assessment” every five years. The purpose of this 
assessment was to provide each discipline the opportunity to examine the effectiveness and 
efficiency of its offerings. All of the academic programs used a common set of criteria to guide 
their reviews. The College’s Curriculum and Instruction (C&I) Committee, via its 
Subcommittee C (Program Review), reviewed the studies and filed a report with the Vice 
President for Academic Affairs. 

There was a high level of frustration from some faculty who participated in this process 
because there were neither incentives nor rewards, particularly in the form of resources, for units 
to complete the self-study. Despite this frustration, however, many faculty saw it as a 
worthwhile activity because it provided them an opportunity to review hard data related to their 
disciplines and to examine ways they might improve their areas. 

In addition, student services units had undergone sporadic reviews in the past few years. 
The Student Services Committee’s Subcommittee 4 (Program Review and Student Services 
Technology) had been less proactive in its efforts to enforce the schedule reviews. The student 
services units that had participated in self studies had found the exercise to be useful, just as the 
academic programs had; and the Student Affairs sector, as a whole, expressed a desire for a 
routinely scheduled, structured program review process that ensured that every student services 
unit would undergo an evaluation. 

Administrative units, such as Fiscal Affairs, Human Resources/Legal Affairs, and 
Institutional Advancement, had never participated in a program review. The 
Superintendent/President expressed a strong desire that the quality and effectiveness of every 
program and services offered by the College be evaluated. The administrative units welcomed 
the opportunity to study their operations, with input from other campus entities. 

The resulting program review process was one that involved all three entities - academic 
programs, student services, and administrative units. Each program, service, and unit is now 
evaluated on both a set of common criteria as well as criteria unique to each entity. A 
procedures guide that incorporated all three areas of the College was prepared and is distributed 
to each unit as it begins its review. 

Design by representative groups. It was important to the campus constituency that the 
process be designed by a group that represented the entire College rather than being a “top- 
down” mandate. The thought was that this would ensure every unit’s buy-in and, therefore, their 
participation in the process. To that end, the initial design committee consisted of 14 people 
representing various organizational sectors (Academic and Classified Senates, C&I Committee, 
Student Services Committee, administrative units) and employee groups (faculty, classified, 
department chairs, deans, vice-presidents, and president). Three people coordinated the design 
process: the director of institutional research; the project director of the Title III grant that 
included program review as a major component; and an external consultant who had significant 
expertise in the area of program review. 
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Relationship to mission. In the early 1990’s, the College’s mission statement had been 
developed as a result of broad-based input from all College constituents. The campus 
community reviewed the mission statement in 1998 and determined that the statement, as 
written, still depicted the College’s commitment to its students and recommended no changes to 
the mission statement. 



Southwestern College Mission 

Southwestern College is committed to meeting the educational goals of its students in an 
environment that promotes intellectual growth and develops human potential. 



When discussions regarding program review began, the group decided that the truest 
measure of the College’s effectiveness lay in determining how effectively it fulfilled this student- 
centered mission. The external consultant helped to cement the process by suggesting that the 
process be titled “Achieving Institutional Mission.” In practice, the mission statement appears 
on all documents related to the program review process, and a logo that depicts the goal of 
“Achieving Institutional Mission” was created and is widely used. 

Program improvement. “The intent of the program review process is to promote 
student-centered educational and service excellence by engaging all college units in self- 
examination and self-improvement.” This statement appears in the opening pages of the 
Southwestern College Universal Procedures Guide as a statement of the College’s commitment 
to use the results of program review for constructive, not punitive, purposes. It is the 
responsibility of each unit undergoing review to point out both the positive areas of the unit and 
the areas that need improvement, along with recommendations as to how to make the needed 
improvements. The information gathered and analyzed by the units conducting their self-studies 
are to be used at every level of the college to aid in planning, decision making, personnel 
development, program improvement, and better utilization of the College’s resources. 
Specifically, program review is designed to: 

• Ensure that all college programs and services are functioning in support of the 
college’s student-centered mission. 

• Promote steady improvement in the quality and currency of all college programs and 
services. 

• Provide evidence of institutional effectiveness at all levels for accreditation. 

• Promote broader understanding of the College’s many programs and services. 

• Facilitate self-analysis of each unit’s functions and its relationship to college goals 
and the internal and external conditions that impact its operation. 

• Note areas of strength and acknowledge accomplishments. 

• Note areas in need of improvement to alert the College to problems in time for 
proactive solutions. 

• Provide a vehicle for information-based, timely, equitable input for budget 
consideration to support development and improvement of all college programs and 
services. 

Related to campus processes. Rather than imposing on the College another project that 
would operate in a vacuum, the campus community insisted that program review be related to 
other ongoing processes. Figure 1 shows the relationship among some of the campus processes. 
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Appendix A shows a more detailed timeline of the interaction of the seven-year program review 
timeline and the overlap of WASC accreditation and the College’s institutional and technology 
planning, budget development, and mandated reporting. It was the desire of the constituents that 
the results of the program review self-studies would inform each of the other processes. 

Figure 1 

Relationship of Southwestern College Mission, 

WASC Accreditation, Institutional Plan, 

Program Review, and Budget Development 




► 



>• 

> 



► 




Link to funding process. A major concern with the previous attempts at program 
review was that there was no incentive for a unit undertaking such a labor-intensive 
introspective. That is, there were neither resources allocated to fund the needs that surfaced 
during the review nor consequences for a unit that chose not to complete a review. The goal of 
the current project was to use the self-study findings to help set the College’s funding priorities: 
Many of the discussions held in the formative AIM Committee meetings centered around 
resource issues. The funding process that was used in the first year of the review was the result 
of these discussions. (See Appendix B.) 

The College Leadership Council (CLC) is a shared governance group compromised of 
faculty, administrators, classified staff, and students. This group is charged with serving as the 
budget steering committee. In particular, the CLC has sole discretion regarding the distribution 
of the State’s block grant monies, earmarked for instructionally related equipment, library 
materials and equipment, and materials that increase the use of modem technology for 
instructional purposes. To accommodate program review while also ensuring that other needy 
units had access to funding, the College’s Office of Institutional Advancement, through its 
Grants and Development Office, was asked to assist the CLC in designing a competitive process 
that would acknowledge units undergoing program review. The resulting process gave 15 bonus 
points to a unit scheduled for program review that submitted a proposal. Of the 30 proposals 
ultimately funded, for a total of $1.7 million, 28 proposals represented units undergoing program 
review. 

In addition to this CLC/block grant funding process, unit administrators were encouraged 
to use the self-study recommendations and associated funding issues in developing their budgets 
for the coming year. In this way, units’ needs that might not have been eligible or appropriate 
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for the CLC/block grant process would have access to resources. This would also afford a long- 
term look at those fiscal needs that would extend beyond one year. 

While this approach was successful in establishing a culture of resources tied to program 
review, it was only an initial step. Recognizing the unstable nature of grant funds and the 
restricted distribution of these block grant funds, the AIM Committee is considering other 
funding alternatives that would ameliorate these constraints. The continuing goal is to fully and 
seamlessly incorporate the results of the self-studies into the College’s budget 
development/funding process, thus ensuring that every academic, student service, and 
administrative unit be assured of some level of funding for its documented needs. 

Schedule equitability. Another issue expressed by the College community was that the 
units scheduled for review be published according to an equitable arrangement. With over 130 
academic, student services, and administrative units that would undertake a program review self- 
study between 2000-2001 and 2004-2005, it was important that both structure and flexibility be 
built into the schedule. Such considerations as length of time since the previous review 
(especially for academic units and for some student services units), size and scope of units 
(particularly such administrative units as Fiscal Affairs, which consists of 14 separate 
components to be reviewed), and whether departments and their associated functions would be 
reviewed simultaneously (such as Student Support Services administrative department along 
with its associated functions of Extended Opportunity Programs and Services, Disability Support 
Services, Student Employment Services/Cooperative Education) were taken into account. The 
resulting schedule spread the reviews and separated the 130 units somewhat evenly across the 
five-year period. 

By the time the College was into the second year of the cycle, all of the units agreed that 
it would be more feasible for the administrative and operational functions to be reviewed in the 
same year. The schedule has been revised for the remaining years of the process as shown in 
Appendix C. For example, the School of Business and Information Systems, along with its 
remaining associated disciplines (Business Administration, Computer Information Systems, 
Computer Literacy, Computer Science, Electronics, Legal, and Travel and Tourism) are all now 
scheduled for review together in 2004-2005. 

Procedures Guide 

As each of the campus issues was addressed and the AIM Committee reached consensus 
on the details of the process, a document was produced to guide each of the units through the 
self-study process. The procedures guide contains the annual timeline, descriptions and 
definitions of data each unit could expect to receive, the criteria for each unit’s review, and all of 
the required forms. The procedures guide is intended to serve as a resource to everyone who 
plays a role in the program review process: the units undertaking a self-study, cognizant 
managers, institutional research, and oversight committees. 

Role of Institutional Research 

The Director of Institutional Research was one of the three people who coordinated the 
development of the College’s program review process. Once the design was in place and 
implementation began, it was the job of Institutional Research to provide information and 
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technical support to the various units undergoing review. These tasks fell into two main 
categories: providing statistical data and coordinating the survey process. Each of those 
components is detailed below. 

Southwestern College (SWC) Data. One of the primary roles of Institutional Research 
(IR) is to provide statistical data for academic programs undergoing review. IR supplies student 
and course characteristics for the five fall semesters preceding the year of the review, for each 
discipline, its associated school and departments, and college totals. The data elements for the 
student characteristics included their major, sex, age group, and ethnic group. The data elements 
for each course included maximum load, enrollment at census, percent load at census, student 
contact hours, full-time instructor equivalent, student contact hours/full-time instructor 
equivalent, and full-time/part-time instructor status. 

Purpose . The statistical data described the unit in terms of student contact and outcomes 
and staff assigned to the unit. The data are used in conjunction with other information to assess 
the quality and effectiveness and level of service for each course. The data also helped faculty 
and administrators respond to the common set of criteria for academic program review. 

Data source . The data used to compile the reports were taken from the College’s 
computer system, Datatel/Colleague. Therefore, the accuracy and completeness of the data for 
each element is, in part, dependent upon the qualities in the College computer records used to 
compile this report. Additionally, data elements involving major, sex, age group, and ethnic 
group are self-reported by the students when they apply for admission to the College. 

Process . The process of analysis for program review during the first year evolved from 
the College’s discipline assessment, a less comprehensive review. The fact that there was 
already a model in place for the statistical analysis helped tremendously because it provided a 
strong foundation for the more extensive study. This was especially important because the first 
year of program review marked a lot of changes for IR in terms of providing data. The most 
significant change was the amount of data provided to the programs. In previous years, programs 
were provided with data for one fall semester, compared to the new requirement of five fall 
semesters. Additionally, the College was undergoing a reorganization of its schools and 
departments. The reorganization moved some programs to other schools, merged some 
programs, and split some into two programs. IR had to accommodate these changes in its 
reporting. 

The first step for IR was compiling the data. As previously mentioned the process had 
been standardized; the data to be provided and the format were already established. Therefore 
there was very little contact with other units on campus with the exception of a few phone calls 
to Academic Services regarding changes from the reorganization of the College. 

Once the data were compiled, IR held training sessions for the School Deans, Department 
Chairs, and Faculty Leads of the programs undergoing review. The primary purpose of the 
training sessions was to explain how to interpret the data. Additionally, since the co-chairs of 
Subcommittee C attended the training sessions, this provided the key players of program review 
with an opportunity to ask questions and receive clarification before beginning to write their self- 
study. 
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Finally, the academic units undergoing review wrote their self-study. The role of IR 
during this step of the process was minimal. Although IR had played a significant role in 
facilitating the design of the program review process, they are not the owners of the process. 

The academic units were encouraged to contact IR with their questions regarding the statistical 
data; but in efforts to reiterate the perception that the review process is “universal,” they were 
referred to various units on campus for all of their other inquiries. 

Product . IR compiled customized data books for each academic unit, school and 
department undergoing review the first year. The second year the customized books were 
eliminated and the data made available on-line. Although the customized books were well 
received and were very successful, assembling them was very labor intensive and time 
consuming. The streamlined on-line data system is much more efficient and is an appropriate 
use of available technology. 

SWC Transfers to San Diego State University (SDSU) Data. An added role of IR is to 
provide statistical transfer data for all transfer programs undergoing review. During the first year 
of program review, IR piloted the transfer study with Geography. During the second year, the 
transfer study was expanded to the programs undergoing review whose leaders completed 
transfer data request forms. IR supplied admission and enrollment, term units and grades, and 
degree completion information of students who had transferred to San Diego State University 
(SDSU) over a six year period. The data elements for admission and enrollment included major, 
transfer GPA, and transfer units accepted. The data elements for term units and grades included 
campus GPA, grades, term GPA, term units attempted, and term units earned. Finally, the data 
elements for degree completion included degree level, degree, final cumulative GPA, time to 
graduation, and graduation rate. 

Purpose . The transfer data tracked Southwestern College transfer students to SDSU and 
determined their performance and outcomes at the individual course level. The data enabled 
academic units the ability to assess transfer patterns. This helped faculty and administrators 
respond to the criterion for academic program review that shows that the discipline is responsive 
to changing conditions within the field. More specifically, faculty and administrators are able to 
address the following: the extent to which courses and programs have been articulated with 
feeder and transfer institutions; the fact that faculty work with transfer institutions to ensure 
currency of course content and standards; and the fact that the discipline is responsive to the need 
for articulation or the integration and sequencing of courses with transfer institutions that require 
these courses. 

Data source . The data used to compile the reports were taken from the San Diego 
Imperial Counties Community College Association (SDICCCA) Data Sharing Consortium 
database. The primary purpose of the Consortium is to share transfer student data within the 
public community colleges and universities in San Diego and Imperial counties. The 
Consortium includes the following member community college districts: Grossmont/Cuyamaca, 
Imperial Valley, Mira Costa, Palomar, Southwestern, and San Diego. The consortium also 
includes these universities: California State University San Marcos; San Diego State University; 
and University of California, San Diego. The goal of the consortium is to provide colleges with 
the ability to analyze transfer patterns of their students for the purpose of program improvement. 
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Process . The process for compiling the transfer data varies from the process for the 
campus data. First, IR met with the Deans, Department Chairs, and Faculty Leads of the 
academic units undergoing review. The purpose of the meetings was to provide a brief history of 
the SDICCCA Data Sharing Consortium and train the participants to complete Transfer Data 
Request Forms. The request forms also enabled academic units to tailor the final report to meet 
their diverse needs. The reasons for the highly individualized reports for each academic unit 
stemmed from the acknowledgement that faculty are the experts in their fields and are more 
familiar with student enrollment patterns and course requirements at the university than IR staff 
are. The Associate Dean of Student Support Services, who oversees the articulation process, is 
available to assist faculty who have questions regarding transfer patterns. 

Once the academic units submitted their request forms, IR compiled the statistical data 
and IR held training sessions for the academic units undergoing review. The primary purpose of 
the training sessions was to explain how to interpret the data. 

Product . IR compiled customized transfer reports for each academic unit undergoing 
review the first year. The report included the statistical analysis of the transfer data request 
forms as well as information pertaining to the College. The customized reports are effective, and 
IR may consider publishing this information on-line in the future. 

Survey Process. A major component of the program review self-studies conducted by 
administrative units and student services is a series of faculty and staff surveys. The office of 
Institutional Research (IR) plays a key role in the development and administration of the surveys. 

As the first step in developing the surveys, IR staff meets individually with 
representatives of the units scheduled for review. During this meeting, the representatives share 
information regarding the services they provide to the campus. From this meeting, IR staff drafts 
a set of survey questions. Depending on the needs of each unit, the questions are designed to 
meet a variety of goals, such as measuring customers’ awareness of the services the unit provides 
and their satisfaction with those services. The information gathered through the survey process 
provides evidence of a unit’s effectiveness and direction for program improvement. 

Once the surveys have been finalized, through continued interaction between the units 
and IR, the IR staff works with the Web Development Team to develop the online survey 
system. This team, comprised of a web technologist, a web designer, and an ASP programmer, 
is responsible for all aspects of the online version of the survey and from the layout to creating 
the database to collect the responses. At the same time that the online system is being 
developed, IR develops an identical paper survey. 

The surveys are available to all employees of the College: full-time and adjunct faculty, 
all classified staff (including confidential and management employees), administrators, and 
governing board members. Every employee receives a personal message from the 
Superintendent/President introducing the surveys and giving them their unique user name and 
password. This message is sent via e-mail to all employees with college-sponsored e-mail 
accounts from an alias account created and maintained only for this project. The remainder 
receive a similar, personalized, printed message. The IR office coordinates the announcement 
process along with periodic follow-ups during the four to six weeks of the survey. 

Several issues must be considered regarding an online survey process. Security is key; 
and to ensure security, every person is assigned a unique user name and password. Only the 
Web Development Team and the three-member IR staff have access to these identifying features 




10 



11 



and use them only to assist people who request help with the process. Another issue is ensuring 
respondents’ anonymity and confidentiality. No attempt is ever made to identify respondents nor 
are results presented in such a way that a single respondent or a group of respondents can be 
identified. Finally, the survey system must be designed in such a way as to ensure that each 
person is allowed to complete the survey only once but that a person can change responses as 
often as desired, keeping only the last response indicated. Each of those issues have been 
successfully addressed in the SWC survey process. 

The hard-copy surveys, on the other hand, present their own issues. The hard-copy 
surveys are produced using Bubble Publishing to create scannable forms. The questions and 
responses are identical to those on the online survey, but the layout is changed to be more space- 
efficient. Because the entire online process is less labor-intensive for IR staff, everyone is 
encouraged to respond online. Hard-copy surveys are sent only to people who specifically 
request them. The packets are numbered, using the software’s preslugging capabilities, and 
color-coded by survey. The results of the paper surveys are incorporated into the online results 
in a seamless merger of paper and online surveys. 

Once the survey process has been completed, IR staff compiles the results. Each unit 
receives two sets of results, compiled by (1) all respondents and (2) users of that unit’s services. 
This process works in this way. The first question on every survey is, “How many times do you 
contact the unit in a typical academic year?” The assumption is that some people will respond to 
a given unit’s survey based on what they have heard about the unit. Their responses are 
important since as a group they help to understand the reputation the unit has on campus. The 
true customers, however, are those who have contacted the unit at least once; those are the ones 
who can provide first-hand information regarding their experiences with the unit’s processes. 

The two analyses given to each unit are based on these two types of respondents. In addition, all 
comments are delivered to each unit exactly as they were written. 

Finally, each unit receives only its own survey results, presented in a customized binder 
and delivered in a meeting between each unit’s staff and IR staff. No unit sees the data or the 
comments received by any other unit. 

Once the responses have been delivered to the units, it is the responsibility of every unit 
to incorporate the results into its self-study and to use them for program changes as needed. 

First-Year Experiences 

At the end of the inaugural year of the universal program review process, the AIM 
Committee and the program review participants reviewed the successes of the year and evaluated 
the areas that needed improvement. As a result, several changes have been made. 

Participation. For the first time, every unit that was scheduled for program review 
completed a self-study and submitted it to the appropriate group for review. However, many of 
the units had difficulty meeting the timeline. The data and survey results from IR were due to 
the units in late September and their final reports were due in late October. Although the data 
and survey results supplied by IR comprised only a portion of the requirements for their self 
studies, many units did not begin writing their reports until they received information from IR, 
giving them only a few weeks to write their reports. Because of this delay, IR will now have 
data and survey results available in the semester preceding units’ scheduled reviews. 
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Funding. The competitive process that was established for distributing the State’s block 
grant funds while acknowledging the units that had participated in program review was very 
successful. Some $1.7 million was distributed to 30 recipients, 28 of whom qualified for bonus 
points based on program review. However, it was recognized that these funds were restricted in 
that administrative units were not eligible to compete for the funds. Also, because these funds 
are essentially grant funds, by their very nature, they cannot be guaranteed from year to year. 
The continuing goal of the AIM Committee is to develop a method of funding the needs that 
surface as a result of program review self-studies. 

Data. In the first year of program review, the SWC dataset provided to each academic 
program was expanded from the single year of data that each had received in the past to a five- 
year history. Each program was provided with a customized binder that contained data for the 
program, its associated department and school, and college totals. This meant that the IR office 
created 35 customized binders, complete with customized dividers, cover pages, and inserts - a 
very expensive and labor-intensive process. The IR office has designed an online data system 
that will allow participants access to their data without the need for paper reports. Faculty will 
attend a single session to train them in accessing and interpreting their data. The session will 
also include an overview of the program review process. 

Survey Process. The initiation of the program review process also marked the first 
campus-wide survey conducted on behalf of student services and administrative units. The 
survey was well received by the campus community, and units used the results in considering 
program improvements. There were challenges, however, in conducting such a survey; most of 
these challenges involved the logistics and mechanics of designing and administering such a 
comprehensive survey. For example, neither the IR staff nor the web team had any previous 
experience with any aspect of a web-based survey, including design, security, or database issues. 
By working through the first survey series together, both IR and the web team were more 
prepared to undertake the next year’s survey project and to make improvements in the survey. 

The paper-based surveys presented another challenge in that while IR staff had been 
involved in many surveys, the design software and the process of setting up a security and 
tracking system to monitor both the web-based and the paper surveys simultaneously were new. 
Following a year of experience, the IR staff have refined the paper system and streamlined the 
interaction between the two systems. 

Procedures Guide. One of the key products that the AIM Committee generated was the 
procedures guide that gathered all of the forms, timelines, definitions, and criteria into a single 
document to assist the units in preparing their self-studies. Because this guide was prepared 
prior to beginning the program review process, it was expected that actual practice might differ 
from what was printed in the guide. Thus, throughout the year, the guidelines were discussed by 
all of the units involved in the process and necessary changes made to the guide. These changes 
were incorporated into a revised procedures guide, to be sent to all units participating in review 
during the next academic year. 



Conclusion 



The purpose of the universal program review process at Southwestern College, as 
designed by the AIM Committee, was to provide an information base for all of the academic 
programs, student services, and administrative units to use as they determine ways to improve 
the programs and services they offer in support of student learning. The mission statement, 
which underscores the student-centered commitment of the college, would provide the 
foundation for evaluation of every sector of the college. The first year of the process has resulted 
in an enthusiasm among each of the campus entities for a process that celebrates their successes 
and while providing at least a portion of the resources they need for improvement. Program 
review is becoming an integrated part of the culture of Southwestern College. 
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Southwestern College 

Achieving Institutional Mission: Universal Program Review 
Appendix A - Program Review Cycle and Relationship to Campus Processes 
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Southwestern College 

Achieving Institutional Mission: Universal Program Review 
Appendix B - Budgetary Process 

(Integration of program review reporting process with the annual budget development process) 
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Achieving Institutional Mission: Universal Program Review 
Appendix C - Six-Year Schedule 
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